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Students'  Responses  to  Open-Ended 
Questions  in  iViath 


Grade  8 


The  conversation  below  is  between  a  driver  and  a  pas- 
senger on  their  way  to  London  by  car  The  picture  shows 
what  they  see  as  they  speak. 


'^\  thtnk  we  need  to  slap  for  gas  before  wo  reach  London^ 


'No,  we  should  tse  aiHghL  T>w  tank  holds  about  8  galtonsX 
.«..«.  U w  „ ^ 


and  \  fitlsd  It  up  yesterday.  We  havent  got  time  to  stop. 


-^How  many  ffiiles  does  this  car  do  to  a  satonf) 

CWeB,  on  the  highway  a  does  about  34  mHes  to  a  ga^nr> 


Do  they  have  to  stop  for  gas?  (Explain  your 
reasoning  carefully.  It  will  count.) 


Major  Concepts/Abilities  Tested 

Recognizing  the  matiiematical  structure  in 
an  unstructured  problem 

Interpreting  grapliical  data 

Identifying  relevant  information 

Integrating  information 

Computational  facility 


The  percentage  of  student  responses  and 
appropriate  examples  are  listed  under  each 
of  the  categories  below. 


*  Adapted  from  a  problem  developed  by  the  Shell  Centre  for 
Mathematical  Education. 


Massachusetts  Department  of  Education 


April  1991 


Massachusetts 


Educational 
Assessment 
Program 


In  the  spring  of  1990  over  9,000  fourth,  eighth,  and  twelfth  grade  students  were 
assessed  using  open-ended  mathematical,  scientific,  social  studies,  and  reading 
concepts.  This  series  of  reports  describes  and  discusses  the  results  of  these 
assessments.  Prepared  by  Elizabeth  Badger  and  Brenda  Thomas. 


Correct  Answers/Methods 

**  22%  Yes,  by  computing  how  far  the  car  can  go  on 
the  amount  of  gas  in  the  tank  (approximately 
119  miles). 


**    4%  Yes,  by  computing  the  number  of  gallons 
needed  to  drive  135  miles  (3.9  gallons) 


*    9%  Yes,  by  approximating  appropriately 

"I  think  they  should  stop.  They  might  have  enough, 
but  it  would  come  close  because  they  have  less  than  4 
gallons  left." 

**    good  answer 
*    acceptable  answer 


Incorrect  Answers/Methods 

21%  Yes;  incorrect  or  no  reason  given. 

"Yes,  they  do  need  to  stop  for  gas  because  they're  going 
60  MPH,  and  they're  going  to  run  out  of  gas  soon." 

"Yes,  because  they  only  get  34  miles  to  the  gallon." 


Comments 

This  question  challenged  students  to  identify  and  coor- 
dinate different  kinds  of  information  in  order  to  come 
to  a  judgment.  In  contrast  to  most  math  "problems" 
where  the  consequences  of  a  wrong  answer  are  merely 
formal,  an  incorrect  decision  here  could  bring  about 
unpleasant  real-life  consequences.  As  such,  it  is  typi- 
cal of  the  kinds  of  simations  in  which  we  typically  use 
mathematics  and  it  requires  the  same  kinds  of  think- 
ing skills.  There  is  an  abundance  of  different  kinds  of 
information,  from  different  sources.  Some  of  the  in- 
formation is  necessary  in  order  to  solve  the  problem; 
some  is  redundant;  some  is  irrelevant.  An  adequate 
solution  comes,  not  from  the  calculations  required, 
but  from  the  ability  to  translate  the  situation  into  some 
simpler  structure.  In  this  case: 

Given  that  a  car  uses  34  miles  to  a  gallon, 

How  tar  will  it  go  on  xx  gallons? 

or 

How  nfiany  gallons  will  it  take  to  go  xx  miles? 

Once  the  structure  of  the  problem  is  recognized,  stu- 
dents are  then  required  to  interpret  the  information 
given  in  order  to  find  the  right  values.  This  proved  to 
be  an  obstacle  for  many.  Although  a  quarter  of  the 
students  were  able  both  to  recognize  the  mathematical 
structure  and  to  integrate  the  various  pieces  of  infor- 
mation given  in  the  picture,  approximately  the  same 
number  misread  the  fuel  gauge  or  did  not  read  it  at 
all. 


10%  No;  assumed  car  had  full  1/2  tank  (4  gallons). 


18%  No;  assumed  car  had  full  tank  (8  gallons). 


15%  Other  incorrect  response. 

"No,  they  probably  don't  have  to  stop  because  London 
is  135  miles  and  the  car  does  about  34  miles  per 
gallon.  They  would  use  up  about  3  gallons  and  have 
extra  gas  left  over." 

1%  Blank 


It  requires  smdents  to  reason  mathematically  in  a 
practical  context.  Aside  from  their  use  in  e\aluating 
students'  understanding,  problems  like  this  carry  a 
message  about  mathematics  itself  Unlike  the  more 
typical  short  answer  problems  that  have  alreadv-  been 
formulated  by  the  teacher  or  textbook  and  are 
primarily  confined  to  the  manipulation  of  numbers  or 
formulae,  problems  such  as  these  illustrate  how  math- 
ematics is  used  in  order  to  structure  the  quantitative 
information  that  permeates  our  everyday  existence.  It 
shows  mathematics  as  a  way  of  simplifying  that  infor- 
mation in  order  to  make  reasonable  decisions.  This 
notion  of  how  mathematics  can  be  used  often  escapes 
students  who  state  that  what  they  learn  in  class  has 
little  utility  in  their  lives. 
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Students'^  Responses  to  Open-Ended 
Questions  in  iVIath 


Grade  8 


The  percentage  of  student  responses  and 
appropriate  examples  are  listed  under  each 
of  the  categories  below. 


On  John's  tenth  birthday,  John's  grandfather  gave  him 
$10.  He  gave  John  $20  on  his  eleventh  birthday  and 
$40  on  John's  twelfth  birthday.  Following  this  pattern, 
John's  grandfather  plans  on  giving  John  $70  on  this 
thirteenth  birthday,  but  John  expects  $80  from  his 
grandfather  on  that  day.  John's  sister  says  that  both 
amounts  could  be  correct. 

Who  Is  right— John's  grandfather,  John,  or  John's 
sister? 


Correct  Answers/Methods 

**  19%  John's  sister:  recognizes  botli 
sequences. 

"John's  sister.  His  grandfather  could  have 
given  him  $80.  Then  it  would  have  been 
like  doubling  it  each  time.  Or  his 
grandfather  could  have  given  him  $70  by 
adding  $10,  $20,  $30  in  that  pattern  to 
each  year  before's  amount." 


Major  Concepts/Abilities  Tested 

Pattern  recognition 
Reasoning  about  relationships 


*  35%  John:  recognizes  sequence. 

"John  is  right,  because  each  year  his 
grandfather  doubles  his  amount  from  the 
last  year." 


Massachusetts 


Educational 
Assessment 
Program 


Massachusetts  Department  of  Education 


April1991 


In  the  spring  of  1990  over  9,000  fourth,  eighth,  and  twelfth  grade  students  were 
assessed  using  open-ended  mathematical,  scientific,  social  studies,  and  reading 
concepts.  This  series  of  reports  describes  and  discusses  the  results  of  these 
assessments.  Prepared  by  Elizabeth  Badger  and  Brenda  Thomas. 


*  11%  Grandfather:  recognizes  sequence. 

"John's  grandfather  because  every  year  he  keeps  ad- 
ding in  tens." 

$10 -$20=  10 
$20  -  $40  =  20 
$40  -  $70  =  30 


**    complete  response 
*    minimal  response 


Incorrect  Answers/Methods 

25%  No  recognition  of  any  sequence. 

"I  think  the  grandfather  is  the  right  one  because  he  is 
the  one  givingjohn  the  money,  so  it's  his  decision." 

"John  is  right.  There  is  a  certain  pattern  that  his 
grandfather  is  using  to  give  him  money  70  wouldn't 
fit  into  it." 


10%  Blank 


Comments 

There  are  three  different  possible  sequences  in  the  situation 
presented.  One  (recognized  by  John)  is  a  doubling  of  the  gift  each 
year,  which  would  result  in  a  $80  gift.  The  other  two  (one  of  which 
is  followed  by  John's  grandfather)  would  result  in  $70.  Whether 
John's  grandfather  arrived  at  this  figure  by  adding  $10  to  the 
increase  ft'om  the  previous  year  or  by  summing  together  the  gifts 
of  all  previous  years  cannot  be  discovered  until  John  reaches  his 
fourteenth  birthday. 

While  19  percent  of  the  students  recognized  at  least  two  of  the 
possible  patterns,  35  percent  seized  upon  the  most  obvious 
doubling  pattern.  Another  28  percent  did  not  recognize  any 
pattern  and/or  described  an  incorrect  one.  Five  percent  of  the 


students,  not  recognizing  the  problem  as  a  mathemati- 
cal one,  stated  that  the  grandfather  could  do  what  he 
liked  with  the  money. 

More  typical  sequence  tasks,  v^diich  call  for  a  single 
solution,  can  suggest  that  mathematical  reasoning 
consists  of  finding  the  one  correct  answer.  However, 
students  should  learn  that  a  few  terms  in  a  pattern  can 
mean  very  little.  Discovering  a  pattern  is  basically  an 
inductive  exercise  in  which  success  is  dependent  upon 
the  quantity  of  the  data.  Problems  such  as  this 
promote  such  thinking. 

Not  only  is  the  recognition  of  sequences  and  patterns 
important  elements  of  inductive  reasoning,  it  lays  the 
groundwork  for  later  work  with  mathematical  func- 
tions. Exploring  different  numerical  patterns  with 
graphs  or  tables  allows  students  an  opportunity  for 
intuitive  understanding  of  the  relationship  between 
variables.  The  need  to  express  that  relationship  leads 
naturally  to  expressions  or  equations.  When  students 
see  algebraic  equations  as  symbolic  ways  of  describing 
an  existing  relationship  between  numbers,  they  are 
better  able  to  recognize  their  generality. 

Nature  provides  a  rich  source  for  the  exploration  of 
patterns,  as  do  number  sequences  themselves.  How- 
ever, students  should  not  only  should  be  exposed  to 
the  patterns  devised  by  others,  but  they  should  be 
encouraged  to  make  up  their  own  sequences.  Playing 
such  games  as  "What's  My  Rule?"  builds  up  a  con- 
fidence with  using  numbers,  as  well  as  the  possibility 
of  discovering  interesting  relationships. 
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students'  Responses  to  Open-Ended 
Questions  in  iVIatli 


N 


Grade  8 


The  Mitchells  took  a  4-hour  car  trip  from  Smithville  to 
Guilford.  The  map  and  graph  below  help  describe  their  trip. 


The  percentage  of  student  responses  and 
appropriate  examples  are  listed  under  each 
of  the  categories  below. 

12:00  -  1:00 


Smithville 


Distance 
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Smithville 
(miles) 
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Use  the  map  and  the  graph  to  describe  what  the 
Mitchells  were  doing  during  each  1 -hour  interval.  Tell 
as  much  as  you  can  about  the  trip  (e.g.,  I(inds  of  roads, 
traffic  jams,  stops,  etc.) 


Correct  Answers/Methods 

***  10%  More  than  one  legitimate  inference, 
but  must  include  rate  of  travel. 

"From  12:00  to  1:00,  the  Mitchells  drove 
60  miles.  They  made  good  time  going 
about  60  miles  an  hour.  No  traffic  jams, 
roads  look  straight  for  the  most  part." 

"The  Mitchells  travelled  at  a  constant 
speed  of  60  miles  per  hour.  Because  they 
were  traveling  at  this  speed,  they  were 
most  likely  driving  on  a  highway" 


Major  Concepts/Abilities  Tested 

Map  reading 

Graph  reading 

Ability  to  relate  information  from  different  sources 


**  46%  At  least  one  legitimate  inference. 

"They  drove  60  miles.    Didn't  have  to 
make  any  stops  and  the  roads  were  clear. 


Massachusetts  Department  of  Education 
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Educational 
Assessment 
Program 


In  the  spring  of  1990  over  9,000  fourth,  eighth,  and  twelfth  grade  students  were 
assessed  using  open-ended  mathematical,  scientific,  social  studies,  and  reading 
concepts.  This  series  of  reports  describes  and  discusses  the  results  of  these 
assessments.  Prepared  by  Elizabeth  Badger  and  Brenda  Thomas. 


They  didn't  seem  to  run  into  any  traffic  jams  because 
of  accidents  or  detours." 


1:00  -  2:00 


*  14%  Minimal  response  or  some  inaccuracies. 

"Between  12:00  and  1:00,  the  Mitchells  traveled  60 
miles." 

*  *  *    very  good  answer 
**    good  answer 
*    minimally  correct  answer 


Incorrect  Answers/Methods 

10%  Attributes  characteristics  of  graph  or 
map  to  road. 

"They  traveled  on  a  straight  road,  going  northeast. 
They  traveled  51  miles." 


15%  Other  incorrect  answer. 

"They  left  Smithville.  Probably  stopped  at  a  restaurant 
for  some  lunch  and  then  headed  off  again." 


Correct  Answers/Methods 

***  9%  More  tlian  one  legitimate  inference, 
but  must  account  for  complete  stop 
and  starting  again  at  same  rate. 

"From  1:00  to  1:30  they  stopped  driving, 
probably  to  get  a  bite  to  eat  and  some  gas 
for  the  car.  At  1:30  they  again  started  to 
travel  at  a  constant  speed  of  60  miles  per 
hour." 


**  29%  At  least  one  legitimate  inference. 

"They  were  still  driving.  They  might  have 
stopped  for  gas  and  at  a  restaurant  to  get 
a  drink  and  use  the  rest  rooms." 

*  24%  Minimal  response  or  some 
inaccuracies. 

"During  this  hour  they  traveled  30  miles. 
They  probably  ran  into  a  lot  of  traffic 
jams." 


4%  Blank 


Comments 

Students  were  most  successful  in  interpreting  this  part  of  the 
Smith's  journey.  The  positive  and  uninterrupted  slope  of  the  line 
suggests  highway  travel,  as  does  the  fact  that  the  road  skirts 
Smithville  without  entering  the  town.  Although  only  a  small 
proportion  of  the  students  explicitly  stated  the  resulting  rate  of  60 
miles  an  hour,  the  graph  depicted  a  steady  rate  of  speed,  and  the 
distance  traveled  was  easy  to  read. 


Incorrect  Answers/Methods 

11%  Attributes  characteristics  of  graph 
or  map  to  road. 

"I  think  that  they  hit  a  detour  or  a  place 
where  they  weren't  supposed  to  go  such  as 
a  forest  because  they  changed  course." 

"They  have  traveled  about  60  miles  and 
have  seemed  to  have  gone  steady  to  the 
right." 


24%  Other  incorrect  answer. 


"They  had  traffic  between  1 :00  and  1 :30.  They  didn't 
get  much  distance,  but  for  the  last  half  hour  they 
cruised  down  the  highway  and  drove  another  60  miles 
in  a  half  hour." 


4%  Blank 


Comments 

The  same  proportion  of  students  who  had  correctly  inferred  that 
the  Mitchells  started  their  journey  by  traveling  on  a  highway  at  a 
rate  of  60  mph,  recognized  that  they  came  to  a  complete  halt  from 
1 :00  to  1 :30,  then  continued  their  journey  at  the  same  rate  as 
before.  However,  most  students  found  the  change  in  the  direction 
of  the  graph  difficult  to  interpret.  Although  some  students  sug- 
gested a  stop,  few  appeared  to  recognize  that  the  graph  depicted 
the  relationship  between  the  two  variables  of  time  and  distance 
and  that  the  horizontal  line  signified  that  one  (distance)  did  not 
increase  in  relation  to  an  increase  in  the  other  (time). 


"From  2:00  to  2:15  they  were  all  right,  but 
from  2:20  to  3:00  they  were  on  an  ugly 
road  with  loads  of  traffic." 

*  21%  Minimal  response  or  some 
inaccuracies. 

"About  30  miles,  hilly  roads,  stops." 

"At  2:00  they  had  gone  90  miles.  At  2:30 
they  had  gone  1 1 1  miles.  At  3:00  they  had 
gone  122  miles." 


2:00  -  3:00 


Incorrect  Answers/Methods 

27%  Attributes  characteristics  of  graph 
or  map  to  road. 

"The  Mitchells  were  driving  at  a  fast 
moving  pace  despite  the  steepness  of  the 
ascent  and  then  the  rugged  steepness." 

"From  2:00  to  3:00,  it  started  out  pretty 
straight.  Then  they  must  have  gotten  onto 
a  secondary  or  a  dirt  road  because  things 
were  pretty  bumpy  for  them." 


Correct  Answers/Methods 

***    9%  More  than  one  legitimate  inference, 
but  must  include  arrival  in  city. 

"They  went  through  Metropolis.  They  probably  got  in 
a  few  traffic  jams  because  it  is  a  large  city  with, 
probably,  a  lot  of  highways.  These  traffic  jams  would 
not  be  bad  though,  and  would  clear  out  rather  quickly " 


**  25%  At  least  one  legitimate  inference. 

"The  Mitchells  started  out  traveling  at  a  constant  speed 
but  seemed  to  get  themselves  into  some  sort  of  a  traffic 
jam." 


14%  Other  incorrect  answer. 

"During  the  third  hour,  they  probably 
stayed  on  the  same  road  with  no  stops. 
Also  no  traffic." 

"They  are  14  miles  away  now.  They  are 
traveling  through  a  meadow  near  the  seas 
or  lakes." 


5%     Blank 


Comments 

In  this  section  of  the  trip,  the  map  contributes  a  great  deal  to 
understanding  what  was  happening  during  the  journey  since  the 
Mitchells'  arrival  at  the  city  was,  undoubtedly,  the  cause  for  their 
unsteady  rate  of  travel.  Most  students  appeared  to  ignore  the  map, 
concentrating  on  the  information  contained  in  the  graph.  About 
a  quarter  of  the  students  recognized  that  the  variability  of  the  slope 
reflected  stop  and  go  traffic,  but  many  concentrated  only  on 
reading  the  graph  (e.g.,  number  of  miles  traveled)  rather  than 
interpreting  it.  The  variability  of  the  graph  also  led  many  students 
to  interpret  the  slope  as  a  depiction  of  the  road  itself.  In  this 
section,  twice  as  many  students  as  before  attributed  charac- 
teristics of  the  graph  to  the  road,  forgetting  that  the  graph  repre- 
sented the  rate  of  travel,  not  its  conditions. 


Incorrect  Answers/Methods 

2 1%  Attributes  characteristics  of  graph 
or  map  to  road. 

"At  about  three-thirty,  the  bumpy  road 
turned  back  into  the  smooth  ride  they  had 
earlier." 

18%  Other  incorrect  answer. 

"More  traffic  jams  and  there  were  holes  in 
the  road.  Some  main  highways  were 
blocked." 


5%     Blanl( 


3:00  -  4:00 


Correct  Answers/Methods 

***    6%  More  than  one  legitimate  inference,  but 
must  include  leaving  the  city. 

"For  this  1/2  hour,  they  were  still  in  Metropolis.  They 
traveled  15  miles.  The  second  1/2  hour,  they  were  out 
of  Metropolis  and  traveled  the  last  15  miles  to  Guil- 
ford." 


**  26%  At  least  one  legitimate  inference. 

"For  the  first  half  hour  it  was  stop  and  go,  but  then  it 
picked  up  and  the  last  half  hour  was  smooth  driving." 


*  24%  Minimal  response. 

"It  took  150  miles  but  the  Mitchells  have  finally 
reached  Guilford  after  4  hours  of  travel." 


Comments 

Students'  responses  to  this  section  were  similar  in 
kind  to  the  previous  section.  Those  who  were  able  to 
use  both  sources  of  information  noted  that  the 
Mitchells  spent  much  of  the  hour  navigating  traffic 
through  Metropolis,  but  for  the  last  20  minutes  of  their 
journey  they  were  again  on  an  open-road,  though 
travelling  at  a  much  slower  rate  than  they  had  done  on 
the  highway. 

In  general,  this  proved  to  be  a  difficult  problem  for 
students.  Students  who  understood  how  to  read  the 
graph  performed  in  a  similar  manner  over  all  four 
intervals.  Those  who  did  not  understand  varied  in  the 
extent  to  which  they  attributed  characteristics  of  the 
graph  or  map  to  the  road.  Many,  undoubtedly,  did  not 
understand  what  the  slope  of  the  graph  signified. 
Others  found  difficulty  in  integrating  two  diff'erent 
types  of  information. 

This  question  was  adapted  fi-om  an  instructional  unit 
on  informal  use  of  functions  which  was  developed  by 
the  Shell  Centre  for  Mathematical  Education.  Its  as- 
sumption is  that  a  familiarity  and  confidence  in  using 
functional  graphs  allows  students  to  develop  a  deeper 
understanding  of  the  kind  of  relationships  that  under- 
lie mathematics. 
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Students'  Responses  to  Open-Ended 
Questions  in  lleading 


Grade  8 


The  citizens  of  Mansfield,  Massaciiusetts  are  debat- 
ing wtietlier  to  aliow  a  Bruce  Springsteen  concert  in 
tlieir  town.  Frequently,  a  person's  argument  is  based 
on  an  unstated  assumption— a  belief  the  person  has 
which  affects  what  the  person  says  or  does. 

The  police  chief  stated  that  he  opposed  the  concert 
because  there  are  not  enough  officers  on  the  police 
force. 

What  is  an  unstated  assumption  behind  the  chief's 
argument? 

is  the  assumption  a  reasonable  one?  Why  or  why  not? 


Abilities  Tested 

Critical  analysis  and  evaluation 

The  percentage  of  student  responses  and  appropriate 
examples  are  listed  under  each  of  the  categories  below. 


Correct  Answers 

**  i2%  Answer  presents  a  valid  assumption 
related  to  the  chief's  argument  and 
evaluates  this  assumption. 

Unstated  assumption:  "That  the  concert 
would  be  wild  and  there  wouldn't  be 
enough  officers  to  restrain  the  crowds." 

Reasonable?  "Yes,  because  Rock  n'  Roll 
concerts  tend  to  be  wild." 

Unstated  assumption:  "That  fights  break 
out  at  concerts  between  teenagers — since 
he's  old  and  thinks  all  teenagers  do 
drugs." 

Reasonable?  "No.  It  is  stereotyping  the 
people  who  have  concerts  by  assuming  too 
much  of  the  teenagers." 

Both  students  correctly  identified  the 
assumption  behind  the  police  chief's  ar- 
gument but  the  quality  of  their  evalua- 
tion differed.  The  student  who 
concluded  that  the  chief's  assumption 
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was  well-founded  referred  to  his  knowledge  of  the 
past  history  of  other  concerts  where  police  had  been 
needed  to  restrain  the  crowds.  The  other  student 
argued  ad  hominem,  suggesting  that  the  police 
chief's  assumption  was  based,  not  on  facts,  but  on 
his  personal  perception  of  teenager  behavior 

15%  Answer  presents  a  valid  assumption  but 
comments  on  the  chief's  argument  instead 
of  evaluating  the  assumption  behind  his 
argument. 

Unstated  assumption:  "There  aren't  enough  officers 
available,  if  a  riot  breal«  out  many  people  could  be 
injured  or  even  killed." 

Reasonable?  "Yes,  you  shouldn't  take  chances  when 
people's  lives  and  well  being  are  at  hand." 

This  student  accepts  the  assumption  (that  police 
control  is  necessary)  as  a  given  and  proceeds  to 
evaluate  the  police  chief's  argument,  which  is  that 
there  are  insufficient  police  for  the  task. 

*  *   fiill  and  complete  response 
*    satisfactory  response 


Incorrect  Answers 

23%  Answer  addresses  the  reasonableness  of  the 
chief's  argument. 

Unstated  assumption:  "The  reason  the  chief  opposed 
the  concert  is  because  of  the  number  of  officers." 

Reasonable?  "Yes.  Safety  is  a  big  factor  and  if  there  is 
not  a  substantial  amount  of  law  enforcers,  then  it  may 
be  a  mistake  to  allow  the  concert  to  take  place.  On  the 
other  hand,  private  security  could  be  hired." 

This  student  explains  why  the  chief  may  be  opposed 
to  the  concert  and  judges  whether  or  not  this  argu- 
ment is  reasonable  without  recognizing  that  the 
validity  of  the  argument  rests  on  whether  or  not  the 
assumption  itself  is  reasonable. 


6%  Answer  accepts  chief's  argument 
and  discusses  how  to  deal  with  it. 

Unstated  assumption:  "He  needs  more 
police  on  this  force." 

Reasonable?  "To  get  police  so  he  can  be 
somewhere  else  helping  others.  That  is 
not  much  to  ask." 


10%   Other  incorrect. 

Unstated  assumption:  "The  reason  he  gave 
wasn't  a  real  good  reason,  it  was  just  an 
excuse." 

Reasonable?  "No.  Bruce  Springsteen  is 
liked  by  many  people  in  Mass  and  it  just 
wouldn't  be  right  to  take  the  concert  from 
him." 

The  student  addresses  the  police  chief's 
argument.  Her  answer  hints  at  the  pos- 
sibility that  the  argument  was  motivated 
by  an  unstated  belief  but  she  fails  to 
identify  that  belief.  Instead,  she 
evaluates  the  consequence  of  the  action 
and  decides  that  it  is  not  justified. 

3%     Blank 


Comments 

The  ability  to  analyze  and  evaluate  arguments  is  an 
important  objective  of  the  social  studies  curriculum; 
however,  it  is  also  a  factor  in  reading  comprehension. 
Unless  students  are  able  to  identifv*  the  unstated  as- 
sumptions that  underlie  argument,  they  cannot 
evaluate  argument  effectively.  However,  this  ability  to 
analyze  and  evaluate  material  goes  beyond  the  critical 
appraisal  of  expository  writing.  It  can  be  equally 
necessary  in  the  understanding  of  literary  characters' 
actions  and  attimdes.  In  all  reading,  it  is  an  important 
aspect  of  comprehending. 


N    Their   Ow 


N 


students'  Responses  to  Open-Ended 
Questions  in  Reading 


Grade  8 


The  percentage  of  student  responses  and 
appropriate  examples  are  listed  under  each 
of  the  categories  below. 


A  very  short  story  is  going  to  be  broken  down  intofour parts. 
Read  the  first  part;  then  answer  the  question  that  follows 
it  Only  after  you  have  answered  the  question  that  follows 
that  part  are  you  logo  on  to  the  next  part 


Part  1 


The  stillness  of  the  morning  air  was  broken.  We 
headed  down  the  bay. 

What  are  some  questions  you  would  want  to  have 
answered  so  that  you  would  understand  what  this 
story  is  about? 


Major  Concepts/Abilities  Tested 

The  ability  to  recognize  the  l<incls  of  information  needed 
to  construct  meaning  from  the  text. 


Correct  Answers 

**  68%  At  least  two  questions  that  reflect 
experience  with  narratives,  e.g., 
questions  about  plot,  characters, 
setting,  title,  genre. 

"What  are  they  planning  on  doing?  Who  is 
in  the  story?" 


*  15%  At  least  one  question  described 
above. 


*    3%  Other  legitimate  questions. 

"What  does  'the  stillness  of  the  morning 
air  was  broken'  mean?  What  does  'headed 
down  the  bay'  mean?" 

"How  was  the  morning  air  broken?" 
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*    9%  Some  combination  of  above. 


Part  2 


*  *    very  good  response 
*    adequate  response 


Incorrect  Answers 

3%  Irrelevant  questions  or  questions  al)out 
insignificant  details. 

"Was  the  bay  pretty?" 


1%  Blank 


:>;^i%%%??¥r-#.%2S%??;??;??s?^??^^^?^ 


The  net  was  hard  to  pull,  the  heavy  sM  and 
strong  tide  made  it  even  more  difticuit  for 
the  girdle.  The  meshed  catch  encouraged 
us  to  try  harder. 

What  do  you  think  is  going  on  in  this  story? 
Why  do  you  think  so? 


.•  ^        .•.•.*  J'  ^^     .•^j'^.'    ^  f/f/^^^^ 


Major  Concepts/Abilities  Tested 

Inferential  comprehension. 


Comments 

Reading  experts  generally  agree  that  reading  involves  more  than 
decoding  the  author's  meaning.  Rather,  readers  construct  their 
ideas  as  they  progress  through  the  text.  The  purpose  of  this  set  of 
questions  was  to  chart  that  progress.  The  passage  is  taken  from 
one  employed  by  Stephen  Norris  and  Linda  Phillips.  ^  The  ques- 
tions attempt  to  measure  how  well  students  build  meaning  for 
themselves. 

This  stress  on  construction  of  meaning  implies  activity.  Readers 
must  be  aware  of  the  kinds  of  information  that  they  need  and  to 
what  extent  their  own  knowledge  is  relevant.  Although  the  descrip- 
tion of  this  process  may  appear  to  be  abstract,  in  the  everyday 
process  of  making  sense  of  the  written  word,  there  are  strategies 
that  help.  One  is  the  practice  of  anticipating  the  kinds  of  infor- 
mation that  will  be  needed  to  make  sense  of  the  passage.  The  first 
question,  which  asked  students  to  list  some  questions  that  would 
help  them  better  understand  the  story  attempted  to  measure 
students'  ability  to  recognize  relevant  kinds  of  information. 


Correct  Answers 

*  79%  Correct  inference  based  on  context 
clues. 

"Fishermen  are  trying  to  pull  a  net  full  of 
fish  onto  the  boat.  I  think  so  because  thev 

4 

are  out  to  sea  pulling  a  net  out  of  the 
water." 

"They  are  fishing.  The  net  told  me  they 
were  trying  to  catch  something  and  the 
'heavy  sea  and  strong  time'  told  me  that 
they  were  in  water." 


Incorrect  Answers 


16%  Correct  inference,  no  reasons  given. 


5%  Incorrect 


^  Norris,  Stephen  and  Linda  Phillips.  (1987)  "Explanations  of 
Reading  Comprehension:  Schema  Theory  and  Critical  Thinking 
Theory."  Teachers  College  Record.  Vol.  89,  2,  281-306. 


1%  Blank 


Comments 

Can  students  use  contextual  clues  to  construct  a  general  meaning 
from  a  text,  despite  the  use  of  unfamiliar  words  (girdie)  and 
phrases  (meshed  catch)?  The  difficulty  of  a  text  can  be  a  function 
of  vocabulary,  but  often  there  are  other  factors,  such  as  the 
complexity  of  concepts  or  the  structure  itself,  that  account  for 
whether  or  not  students  understand  the  meaning.  This  question 
measured  students  ability  to  use  the  context  to  overcome  difficult 
vocabulary.  Almost  80  percent  of  the  students  were  able  to  do  this 
with  justification.  Another  16  percent  gave  a  correct  inference, 
although  they  were  unable  to  state  how  they  came  to  their  con- 
clusions. 


Those  are  fish.  He  started  heading  north 
toward  them,  hoping  to  accomplish  the 
task." 

*  22%  Offers  a  reasonable  interpretation  of 
quintets  only.  (Of  this  22%,  one 
percent  gave  an  accurate  definition 
ofquintels.) 


*  11%  Offers  a  reasonable  interpretation  of 
mares*  tails  only.  (Of  this  11%,  six 
percent  gave  an  accurate  definition 
of  mares' tails.) 


Part  3 


With  four  quintels  aboard,  we  were  now  ready  to 
leave.  The  skipper  saw  mares'  tails  in  the  north. 

Explain  the  meaning  of  this  part  of  the  story. 


Major  Concepts/Abilities  Tested 

Inferring  the  meaning  of  words  from  context. 


Correct  Answers 


Incorrect  Answers 

14%  Focuses  on  "ready  to  leave";  ignores 
quintels  or  mares'  tails. 

"The  meaning  of  this  part  of  the  story  is 
that  now  that  the  ship  had  gotten  what  it 
had  come  for,  it  was  now  ready  to  leave  and 
get  some  more  things.  The  skipper  wants 
to  get  some  more  stuff  for  his  luxury." 


12%  Other  incorrect  (e.g.,  repeats  story). 

"Four  quintels  were  aboard  a  ship  and  now 
the  ship  was  ready  to  leave.  The  skipper 
saw  mares'  tails  in  a  northern  direction." 


**  30%  Offers  a  reasonable  interpretation 
ofquintels  and  mares'  tails. 

"The  crew  had  caught  a  certain  number  of  fish  and 
they  were  read  to  head  in.  The  skipper  of  the  boat  saw 
whales'  tails  coming  out  of  the  water  in  the  north." 

"I  suppose  the  four  quintels  are  people  or  workers. 
Now  since  they  were  there,  they  were  ready  to  go.  The 
skipper,  or  boat  driver,  saw  mares'  tails  to  the  north. 


11%  Blank. 


Comments 

Although  a  few  students  knew  that  a  quintel  is  a 
measure  of  weight,  and  mares'  tails  is  a  type  of  cloud 
formation,  most  students  deduced  the  meaning  of  the 
words  from  the  context.  If  their  responses  were 
reasonable  within  the  context  of  the  sentence  (e.g.,  the 


quintels  are  people  or  cargo;  the  mares'  tails  are  fish),  they  were 
credited.  Those  who  did  not  attempt  an  interpretation  of  the  terms 
were  given  no  credit  because  it  was  impossible  to  explain  this  part 
of  the  story  without  making  some  inference  about  the  meaning  of 
the  words.  Overall,  63  percent  of  the  students  made  some 
reasonable  attempt  to  interpret  one  or  both  of  the  unfamiliar 
words.  The  others  either  ignored  the  terms  or  defined  them  in  a 
manner  that  made  little  sense,  given  the  setting  of  the  story. 


Incorrect  Answers 

11%  Answer  analyzes  context  clues,  but 
does  not  connect  activity  to  prepar- 
ing fish. 

7%  Other  incorrect 


6%  Blank 


Part  4 


We  tied  up  to  the  wharf.  We  hastily  grabbed  our 
prongs  and  set  to  work.  The  splitting  was  done  by  the 
skipper.  The  boys  did  the  cutting  and  gutting. 

Explain  In  as  much  detail  as  possible  what  you  think 
is  happening  In  this  part  of  the  story.  Include  in  your 
explanation  what  you  think  the  words  "splitting,  cut- 
ting, and  gutting"  mean. 


Comment 

As  in  the  case  of  the  previous  question,  approximately 
two-thirds  of  the  students  were  able  to  interpret  the 
meaning  of  the  vocabulary  and  infer  the  activity 
described  in  the  text.  There  was  a  greater  tendency 
on  the  part  of  average  and  poorer  readers  to  con- 
centrate on  the  vocabulary  itself,  interpreting  "split- 
ting, cutting,  and  gutting"  without  any  reference  to  the 
setting  of  the  story. 


Major  Concepts/Abilities  Tested 

Inferential  comprehension 


Correct  Answers 

**  68%  Cleaning  or  preparing  the  fish;  explains 
'^splitting,  cutting,  and  gutting." 

The  boat  tied  up  to  some  dock.  They  got  their  equip- 
ment and  started  cleaning  out  the  fish.  The  splitting 
(cutting  in  halO  was  done  by  the  captain.  The  cutting 
(chopping  off  the  head  and  tail)  and  gutting  (cleaning 
out  of  the  insides)  was  done  by  the  boys. 


7%  As  above,  does  not  explain  vocabulary. 


General  Comments 

The  purpose  of  this  set  of  questions  was  to  model  a 
sequence  of  questions  that  might  be  asked  of  students 
as  they  read  a  narrative  or,  preferably,  be  asked  by  the 
students  themselves.  Obviouslv,  the  format  is  artifi- 
cial.  It  is  not  suggested  that  the  act  of  reading  be 
continually  interrupted  by  axtemal  questioning.  How- 
ever, interpretations  of  text  cannot  be  used  by  them- 
selves in  judging  reading  quality  because  people 
reading  equally  well  can  arrive  at  different  interpreta- 
tions. What  is  important  is  the  support  that  people 
offer  for  their  interpretations.  Consequently  it  is  use- 
ful for  teachers  to  constantly  check  the  understanding 
of  their  students,  as  well  as  for  students  to  learn  to 
monitor  their  own  understanding.  Misinterpretations 
of  ambiguous  phrases  and  vocabular)'  can  quickly  lead 
to  larger  misconceptions.  If  reading  is  a  process  of 
actively  constructing  meaning  from  the  text,  students 
should  learn  how  to  make  sure  that  the  meaning  which 
they  construct  is  reasonable. 
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Students^  Responses  to  Open-Ended 


ions  in 


lence 


Grades  4  &  8 


Different  birds  live  in  different  areas  or  habitats  (such  as 
woods,  lakes  marshes  or  open  areas).  They  eat  different 
foods  (such  as  fish,  seeds,  insects,  or  small  animals).  Birds 
with  webbed  feet  swim  well  and  live  on  or  near  water 
Hummingbirds  have  long,  thin  beaks  which  th^  use  to 
draw  nectar  from  flowers. 


Look  at  the  pictures  of  the  birds  and  notice  how  they 
are  different.  What  do  you  thinic  is  the  main  habitat 
and  the  main  food  of  each?  Write  your  answers  in  the 
spaces  to  the  right  of  each  picture.  Also  explain  why 
you  chose  the  answer  you  did  for  each  bird. 


Major  Concepts/Abilities  Tested 

Life  science 
Adaptation 
Observation  skills 
Functions  of  characteristics 
Diversity  of  life 
Habitats 


The  percentage  of  student  responses  and 
appropriate  examples  are  listed  under  each 
of  the  categories  below. 


Correct  Answers 


Gr4    Gr8 


***    9%  14% 


Relates  appropriate  character- 
istics to  BOTH  habitat  and 
food.  (Can  be  the  same  or 
different  characteristics  but 
must  mention  both  habitat 
and  food.). 


Massachusetts 


Educational 
Assessment 
Program 
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In  the  spring  of  1990  over  9,000  fourth,  eighth,  and  twelfth  grade  students  were 
assessed  using  open-ended  mathematical,  scientific,  social  studies,  and  reading 
concepts.  This  series  of  reports  describes  and  discusses  the  results  of  these 
assessments.  Prepared  by  Brenda  Thomas  and  Elizabeth  Badger. 


"I  think  the  reason  this  bird  lives  in  a  high 
branch  is  because  he/she  has  sharp  nails  that 
grip  into  the  tree  and  the  bird's  feet  are  shaped 
that  way.  I  think  the  bird  eats  small  rodents 
because  it  can  swoop  down  and  grab  it  with  its 
sharp  claws."  (grade  4) 


4%     4%    Mentions  characteristics  but 
does  not  relate. 

"This  owl  has  very  long,  sharp, 
powerful  claws  and  legs.  The  wings 
are  powerful  too.  (grade  8)" 


**  13%  25%    Relates  1  or  more  physical  character- 
istics to  EITHER  habitat  or  food. 

"This  bird  has  long  nails  to  catch  the  animals 
and  it's  beak  is  tilted  inward  to  eat  the  animals 
better."  (grade  8) 

(Although  the  student  mentions  two  charac- 
teristics, they  both  relate  to  food.) 


*    6%     2%    Relates  1  or  more  physical  character- 
istics to  habitat  and/or  food  but 
relationship  is  VAGUE. 

"He  eats  rodents  because  look  at  the  feet  they 
were  made  to  grip  and  lock,  (grade  8)" 

(Although  there  is  a  suggestion  that  the  claws 
are  for  catching  and  grasping  rodents,  it  has 
not  been  made  very  clear.) 


*  *  *    very  good  answer 
**    good  answer 
*    minimally  correct  answer 


3%  5%  Relates  physical  character- 
istics not  evident  from  pic- 
ture to  habitat  and/or  food. 

"This  bird  us  dark  and  has  very  dull 
colors.  It  is  its  camouflage  and 
when  a  victim  comes  along,  the 
mouse  probably  scatters  and  the 
bird  dives  and  catches  him." 
(grades) 


3%     5%    Uses  "because  it  looks  like"  as 
a  reason. 

"His  nose  looks  like  he  would  eat 
insects.  To  me  he  looks  like  a 
woods  kind  of  bird."  (grade  4) 


7%     4%    Other  incorrect  response. 

"He  eats  caterpillars  because  it 
shows  him  sitting  on  a  tree  and 
trees  usually  grow  in  the  woods  and 
caterpillars  crawl  on  trees." 
(grade  4) 


Incorrect  Answers 


3%     8%    Blank 


51%  35%    Gives  FACTS  (not  necessarily  correct) 
about  bird  shown  (prior  knowledge). 

"Owls  live  in  the  woods  and  come  out  at  night 
for  food  and  such.  I  read  a  book  about  on  owls 
and  found  out  about  them,  (grade  4) " 


Comments/Owl 

It's  clear  from  the  results  that  students  learn  a  lot 
about  owls  and  feel  compelled  to  share  their 
knowledge.  The  owl  is  the  only  bird  of  the  four  given 
that  elicited  such  a  large  number  of  students  respond- 
ing in  this  way.  It  could  be  that  the  owl  is  so  interesting 


and  unique  and  such  a  common  cultural  symbol  that  students 
know  more  discrete  facts  about  them.  It  is  possible  that  students 
could  have  incorporated  prior  kno^edge  into  a  correct  response. 
Many  students  did.  For  example,  if  a  student  knows  that  owls  eat 
mice,  then  he  or  she  may  determine  that  the  claws  are  large  for 
tearing  mice  simply  from  knowing  what  owls  eat  and  observing 
the  picture  of  the  owl.  Fewer  students  left  this  bird  blank,  indicat- 
ing that  they  have  some  kno\\iedge  about  o\^is.  However,  it  seems 
as  though  those  blanks  were  picked  up  in  the  prior  knowledge 
category 


**  17%  24% 


observe  a  characteristic  and 
make  a  judgement  based  on  that 
characteristic.) 


Relates  1  or  more  physical 
characteristics  to  EITHER 
habitat  or  food. 

"They  have  pointed  beaks  to  get 
seeds  from  the  ground."  (grade  4) 


*    9%    3% 


Correct  Answers 


Gr4    Gr8 


***    5%  13% 


Relates  appropriate  characteristics  to 
BOTH  habitat  and  food.  (Can  be  the 
same  or  different  characteristics  but 
must  mention  both  habitat  and  food.) 

"The  beak  looks  like  it  digs  worms  from  the 
ground  because  of  the  pointed  end  and  the  feet 
are  somewhat  webbed  and  looks  like  it  could 
walk  on  mud  of  marshes."  (grade  8) 

(Although  the  bird  did  not  have  webbed  feet, 
the  picture  could  been  interpreted  as  such. 
The  student  received  full  credit  for  the 
response  because  he/she  was  able  to  make 
the  appropriate  inferences.  We  were  not 
looking  for  correctness  of  response  regard- 
ing what  the  actual  bird  ate  and  where  it 
lived;  we  were  assessing  students'  ability  to 


*** 


Relates  1  or  more  physical 
characteristics  to  habitat 
and/or  food  but  relationship 
is  VAGUE. 

"This  bird  has  webbed  feet  and 
there  are  many  insects  near  mar- 
shes." (grades) 

(There  is  no  explicit  relationship 
between  webbed  feet  and  insects; 
the  inference  being  that  a  bird 
with  webbed  feet  would  live  near 
the  water,  and  because  insects 
live  near  the  water  the  bird  would 
eat  insects.) 


very  good  answer 
**    good  answer 
*    minimally  correct  answer 


Incorrect  Answers 

28%  21%    Gives  FACTS  (not  necessarily 
correct)  about  bird  shown 
(prior  knowledge). 

"This  bird  is  a  robin  and  usually 
found  in  the  wide  open  area;  for 
example,  your  yard."  (grade  8) 


5%    6%    Mentions  characteristics  but  does  not 
relate. 

"He  doesn't  have  claws  or  a  short  fat  beak. 
Instead  it  has  a  long  skinny  beak  with  webbed 
feet."  (grade  4) 


8%     9%    Relates  physical  characteristics  not 

evident  from  picture  to  habitat  and/or 
food. 

"I'd  say  it's  main  habitat  is  in  a  tree  because 
this  is  a  rather  small  bird  and  needs  a  warm 
and  safe  place  for  itself  and  its  babies.  I'd  say 
it's  main  food  is  worms  or  insects  because  it  is 
a  small  bird  and  would  not  be  able  to  kill  or 
swallow  mice  like  an  owl."  (grade  4) 

(Although  it  may  be  prior  knowledge  that 
leads  the  student  to  believe  that  the  bird  is 
small,  the  picture  does  not  indicate  the  bird's 
size.  In  the  pictures,  the  largest  bird,  the 
heron,  actually  appears  to  be  the  smallest 
When  dealing  with  size,  there  are  some  size 
qualities  that  we  do  accept,  for  example, 
students'  responses  mentioning  the  heron's 
long  beak  are  accepted  as  correct  because, 
in  relation  to  the  rest  of  the  heron 's  body,  the 
beak  is  very  long.) 


Comments/Robin 

About  the  same  number  of  students  answered  this 
question  correctly  as  did  the  previous  question.  Al- 
though the  robin  is  as  familiar  to  most  students  as  the 
owl,  fewer  students  answered  in  the  category  of  prior 
knowledge.  It  could  be  that,  without  seeing  the  colors 
of  the  birds,  students  did  not  recognize  the  robin. 
There  may  have  been  some  confusion  for  some  stu- 
dents who  recognized  the  robin,  but  then  saw  the 
webbed  feet  and  could  not  adjust  their  prior 
knowledge  of  robins  to  what  they  saw  in  the  picture. 


Correct  Answers 


Gr4    Gr8 


8%     8%    Uses  "because  it  looks  like"  as  a  reason. 

"This  bird  looks  like  it  could  be  seen  anywhere 
and  live  comfortably  at  any  of  the  locations.  It 
also  looks  like  it  eats  worms  and  other  insects." 
(grade  8) 


13%     5%    Other  incorrect  response. 

"This  bird  has  eyes  on  each  side  of  his  head  so 
he  or  she  can  see  things  coming  toward  him. 
That  is  for  protection."  (grade  4) 


***    8%  17% 


Relates  appropriate  character- 
istics to  BOTH  habitat  and 
food.  (Can  be  the  same  or 
different  characteristics  but 
must  mention  both  habitat 
and  food.) 

"It  probably  lives  in  tree  trunks 
because  it  can  dig  its  claws  into  the 
side  of  the  tree  to  build  its  nest 
Also,  I  think  it  eats  bugs  because  it 
has  a  long  beak  that  it  can  put  into 
a  tree  and  pull  out  bugs."  (grade 
4) 


7%  11%    Blank 


**  17%  24%    Relates  1  or  more  physical  character- 
istics to  EITHER  habitat  or  food. 

"This  bird's  long  beak  looks  good  for  reaching 
into  the  home  of  insects  because  it's  long  and 
narrow."  (grade  4) 


*    7%    3%    Relates  1  or  more  physical  character- 
istics to  habitat  and/or  food  but 
relationship  is  VAGUE. 

"The  woodpecker  likes  to  carve  out  wood  from 
the  trees  with  its  sharp  beak  and  finds  and  eats 
the  insects."  (grade  8) 

(The  student  response  is  fairly  clear,  but  the 
relationship  between  the  characteristic  and 
the  food  is  very  vague,  bordering  on  incor- 
rect.) 


*  *  *    very  good  answer 
**    good  answer 
*    minimally  correct  answer 


7%    5%    Relates  physical  characteristics 
not  evident  from  picture  to 
habitat  and/or  food. 

"Since  I  have  not  seen  a  bird  like 
this  around,  I  suspect  it  hides  itself 
in  someplace  where  it  can't  be  seen 
easily — the  woods.  It  is  a  smallish 
bird  so  I  would  think  it  eats  in- 
sects." (grades) 


7%    8%    Uses  "because  it  looks  like"  as 
a  reason. 

"He  looks  like  he  is  too  small  to  eat 
meat."  (grade  4) 

10%     5%    Other  incorrect  response. 


"This  bird  looks  for  a  place  where 
he  can  see  more  clearly."  (grade  8) 


8%  13%    Blank 


Incorrect  Answers 

29%  20%    Gives  FACTS  (not  necessarily  correct) 
about  bird  shown  (prior  knowledge). 

"The  woodpecker  pecks  on  trees  in  the  woods 
to  get  insects  out  of  the  trees." 

(This  answer  is  very  similar  to  the  third 
response  above,  but  there  is  no  mention  of 
the  beak  or  its  qualities.) 


Comments/Woodpecker 

Student  responses  here  are  very  similar  to  those  of  the 
robin.  It  appears  that  the  more  usual  looking  the  bird, 
the  less  students  have  to  say  about  them.  Compare 
the  results  of  the  robin  and  woodpecker  with  those  of 
the  heron  which  follow. 


8%    6%    Mentions  characteristics  but  does  not 
relate. 


"He  has  sharp  claws  and  a  pointed  beak." 
(grade  8) 


Correct  Answers 


(Here  there  is  an  inference  of  the 
relationship  between  webbed  feet 
and  fish  as  food  although  it  is  not 
well  elaborated.) 

"Because  he  has  long  legs  you 
would  probably  see  him  near  the 
water  so  he  would  probably  eat 
fish."  (grades) 

(There  is  no  indication  as  to  why 
long  legs  would  make  the  bird  a 
water  bird.) 


Gr4    Gr8 

***  14%  34%    Relates  appropriate  characteristics  to 
BOTH  habitat  and  food.  (Can  be  the 
same  or  different  characteristics  but 
must  mention  both  habitat  and  food.) 

"The  long  legs  can  hold  him  above  the  marshes 
and  the  long  pointed  beak  is  ideal  for  stabbing 
fish."  (grades) 


**  20%  22%    Relates  1  or  more  physical  character- 
istics to  EITHER  habitat  or  food. 

"This  bird  has  long  legs,  a  long  neck  and  a  long 
beak  that  could  allow  him  to  walk  in  the  water 
and  dunk  his  head  into  the  water  to  get  fish." 
(grade  S) 

(Although  three  separate  characteristics  are 
mentioned,  the  student  only  discusses  them 
in  relation  to  food.) 


*** 


very  good  answer 
**    good  answer 
*    minimally  correct  answer 


Incorrect  Answers 

29%  16%    Gives  FACTS  (not  necessarily 
correct)  about  bird  shown 
(prior  knowledge). 

"It's  a  flamingo  and  flamingoes  live 
in  marshlands  and  marshlands  are 
mostly  bugs  and  fish."  (grade  4) 


8%     5%    Mentions  characteristics  but 
does  not  relate. 

"It  has  long  webbed  feet,  a  long, 
sharp  beak  and  a  long  neck." 
(grades) 


*    7%    3%    Relates  1  or  more  physical  character- 
istics to  habitat  and/or  food  but 
relationship  is  VAGUE. 

"They  have  webbed  feet  and  if  they  live  near  or 
on  water  then  they  are  most  likely  to  eat  fish." 
(grade  4) 


5%     1%    Relates  physical  characteristics 
not  evident  from  picture  to 
habitat  and/or  food. 

"He  is  big  and  is  easily  able  to  hunt 
small  animals."  (grade  8) 


5%     2%    Uses  "because  it  looks  like"  as  a  reason. 

"This  bird  loolc  like  a  bird  from  the  jungle  who 
eats  fish."  (grade  4) 


6%     4%    Other  incorrect  response. 

"This  is  the  ancestor  of  the  swan."  (grade  8) 


8%  13%    Blank 


Comments/Heron 

At  both  grade  levels,  students  were  most  successful  when  writing 
about  the  heron.  Approximately  ten  percent  more  of  the  fourth 
graders  and  twenty  percent  more  of  the  eighth  graders  were  able 
to  answer  this  question  successfully  than  were  able  to  answer  the 
other  birds  given;  owl,  robin  and  woodpecker.  What  is  it  then 
about  the  heron  that  lends  itself  to  better  responses?  One  answer 
may  be  that  with  its  long  legs,  neck  and  beak,  the  characteristics 
are  far  more  exaggerated  so  students  are  able  to  more  easily  make 
connections  about  adaptations.  Another  might  be  that  the  bird  is 
far  more  exotic  and  therefore  less  familiar  to  New  England  stu- 
dents. The  owl  carries  cultural  baggage,  the  robin  and  woodpeck- 
ers are  both  fairly  mundane,  but  the  heron  is  not  an  everyday  bird. 


Comments 

The  responses  to  this  question  suggest  that  although 
many  students  are  able  to  make  inferences  based  on 
observations,  a  large  number  have  problems  with  this 
type  of  thinking.  It  is  a  habit  of  mind  rather  than  a 
piece  of  discrete  information.  One  way  of  opening 
students  to  this  type  of  thinking  would  be  to  begin  with 
a  question  rather  than  with  a  fact.  For  example,  these 
questions,  "Why  does  the  owl  have  such  strong,  sharp 
claws?"  and  "How  do  the  owls'  claws  affect  how  they 
survive?"  open  the  door  to  a  more  unified  and  sys- 
tematic way  of  thinking  and  looking  at  something.  By 
connecting  the  structure  of  the  animal  with  its  environ- 
ment and  diet,  we  can  show  the  interrelatedness  of  all 
species.  Students  will  eventually  come  around  to  the 
desired  conclusion,  but  it  will  be  through  a  series  of 
observations  and  discoveries  rather  than  memoriza- 
tion and  recitation.  This  type  of  subject  leads  to  a 
discussion  of  adaptation  and  how,  over  the  genera- 
tions, species  carried  characteristics  that  enabled 
them  to  survive. 

Another  way  of  getting  students  to  examine  function 
and  form  is  to  ask  them  to  draw  a  bird  that  lives  in  the 
water  and  eats  fish,  as  opposed  to  a  bird  that  lives  near 
the  water  and  eats  plankton.  This  method  would 
direct  students  to  forms  of  adaptation. 


N    Their   Ow 


N 


Students'  Responses  to  Open-Ended 
Questions  in  Science 


Grade  8 


Roger  inflated  a  balloon,  tied  the  end  closed  and  put 
it  into  the  refrigerator.  What  happened  to  the  balloon? 
Did  it  get  larger,  get  smaller,  or  stay  the  same?  Ex- 
plain. 


Correct  Answers 

***  3%     Smaller,  with  explanation  at 

molecular  level  including  that  the 
molecules  slow  down  and  move 
closer  together  as  the  temperature 
decreases. 


**  4%     Smaller,  cold  causes  molecules  to 
slow. 


Major  Concepts/Abilities  Tested 

Physical  science 

Temperature  and  states  of  matter 

Particulate  nature  of  matter 


The  percentage  of  student  responses  and  appropriate 
examples  are  listed  under  each  of  the  categories  below. 


"It  probably  got  smaller  because  the  air  is 
cold  and  the  molecules  would  stop 
moving." 

**  17%   Smaller,  cold  causes  molecules  to 
contract. 

"It  got  smaller  because  the  colder  a  gas 
gets,  the  more  it  contracts.  The  molecules 
of  air  inside  the  balloon  drew  closer  to 
each  other  taking  up  less  space." 


*  13%   Smaller,  rubber  or  other  unnamed 
substance  contracts  when  cold. 
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"The  balloon  gets  smaller  if  you  put  it  in  the 
refrigerator  because  in  heat  it  gets  larger  and  that 
means  in  the  cold  it  gets  smaller." 

(This  response  states  that  things  contract  in  the  cold 
without  discussing  why  they  contract.) 


*    5%  Smaller,  air  escaped  balloon. 

"The  balloon  got  smaller.  Most  things  contract  as  they 
get  cold.  The  rubber  of  the  balloon  would  get  smaller 
forcing  the  air  out.  They  balloon  may  have  a  hole  in 
the  first  place  letting  oxygen  seep  out  making  it  easier 
for  the  balloon  to  contract." 

(This  rather  logical  response  deals  with  the  balloon 
as  a  whole  rather  than  its  constituent  substances.) 


fore  the  balloon  would  eventually  expand 
so  large  it  could  pop.  This  is  evident  when 
one  puts  a  soda  bottle  in  the  freezer  for  an 
extended  period  of  time.  The  bottle  will 
burst  due  to  the  pushing  out  of  the 
molecules." 

(Unfortunately,  the  student  used  the  ex- 
ample of  water  as  an  example  for  what 
happens  when  a  substance  gets  cold.) 

"The  balloon  got  larger  because  it  ex- 
panded. Now  if  the  balloon  was  deflated 
and  tied  at  the  end,  it  would  stay  the  same. 
If  there  water  in  the  balloon,  I  assume  it 
would  get  smaller." 


8%  Blank 


Incorrect  Answers 

24%  Smaller,  no  explanation  or  incorrect 
explanation. 

"It  got  smaller  because  the  evaporation  in  the 
refrigerator  made  it  smaller  and  because  there  is  no 
air  or  oxygen  in  it." 

(Although  the  student  is  partially  correct  here  in- 
sofar as  the  balloon  did  get  smaller,  she  does  not 
receive  credit  for  the  response  since  her  explana- 
tion indicates  no  understanding  of  the  effects  of 
temperature  on  a  substance.) 


14%  Same. 

"It  would  stay  the  same.  The  temperature  wouldn't 
change  the  balloon  because  it  is  already  closed." 


11%  Larger. 

"The  balloon  got  larger.  Cold  air  expands  and  pushes 
the  molecules  farther  away  from  one  another.  There- 


*  *  *    very  good  answer 
**    good  answer 
*    minimally  correct  answer 


Comments 

Almost  half  the  students  (42%)  were  able  to  respond 
with  some  degree  of  success  to  this  question.  .\Imost 
a  quarter  (24%)  answered  that  the  balloon  got  smaller 
but  did  not  explain  their  answer  or  gave  an  incorrect 
explanation. 

This  type  of  response  points  to  the  advantage  that 
open-ended  testing  has  over  multiple  choice  testing 
where  the  selection  of  the  option  "smaller"  may  have 
been  considered  correct.  Here  we  do  not  give  credit 
to  those  responses  that  did  not  include  an  explanation 
or  gave  an  incorrect  explanation,  and  we  see  that  it  is 
a  sizable  number  of  students  who  respond  in  that 
manner.  The  purpose  of  this  question  was  to  go 
beyond  the  one-word  answer  to  get  at  students'  con- 
ceptual understanding.  The  responses  to  this  question 
are  similar  to  a  Grade  Four  question  in  which  students 
were  asked  to  explain  how  water  could  create  a 
canyon,  and  22%  responded  with  the  word  erosion 
with  no  explanation  or  an  incorrect  one. 


N    Their   Ow 


N 


Students^  Responses  to  Open-Ended 
Questions  in  Science 


Grade  8 


Here  is  how  a  scientist  often  works: 

The  scientist  observes  something  happening. 

The  scientist  then  thinks  of  a  hypothesis  or  possible 
explanation  of  why  or  how  it  happened. 

The  scientist  designs  or  plans  an  experiment  to  test  the 
hypothesis  to  see  if  it  is  likely  to  be  true. 

When  conducting  the  experiment,  the  scientist  gathers  data 
or  information  which  the  scientist  can  then  analyze  to  help 
draw  a  conclusion  about  the  hypothesis. 


Suppose  you  made  the  observation  statecf 
below.  In  the  spaces  below  the  observa- 
tion, give  a  reasonable  hypothesis  that 
might  explain  the  observation.  Then 
describe  an  experiment  you  would  conduct 
to  test  you  hypothesis.  Tell  what  data  you 
would  gather  if  you  were  to  actually  conduct 
your  experiment. 

Observation:  At  the  end  of  the  marking 
period,  the  students  in  Mr.  Smith's  eighth 
grade  English  class  got  lower  grades  than 
students  in  Mrs.  Jones'  eighth  grade 
English  class. 


Major  Concepts/Abilities  Tested 

Process  skills 
Scientific  inquiry 
Experimental  design 
Use  of  controls 
Central ity  of  evidence 
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The  percentage  of  student  responses  and  appropriate 
examples  are  listed  under  each  of  the  categories  below. 


Correct  Answers 

***  14%  Hypothesis  is  clearly  stated  and  feasible;  exper- 
iment addresses  and  will  test  hypothesis; 
relevant  variables  controlled  (if  necessary); 
explicit  and  clear  description  of  experiment. 

Hypothesis 

"Mr.  Smith  gives  a  much  more  difficult  test  than  Mrs. 
Jones,  but  her  tests  may  be  a  little  too  easy.  And  when 
the  grades  at  the  end  of  the  term  come  out,  the  two 
classes  have  very  different  grades." 

Design  of  Experiment 

"Give  the  teachers  the  same  information  to  teach  and 
then  have  them  make  up  tests  on  that  information. 
Each  teacher  should  give  their  tests  to  the  other  class 
so  that  Mrs.  Jones'  class  is  taking  the  test  that  Mr. 
Smith  wrote  and  Mr.  Smith's  class  is  taking  Mrs.  Jones' 
test.  Also  have  the  two  classes  take  a  third  test  that 
someone  else  made  up  on  the  material  so  that  we  can 
see  how  they  do  on  another  test.  If  Mr.  Smith's  class 
gets  higher  grades  with  Mrs.  Jones  test,  and  if  Mrs. 
Jones  class  gets  lower  grades  than  Mr.  Smith's  then  the 
hypothesis  is  right.  If  the  grades  are  still  like  that  then 
they  should  move  onto  a  different  idea." 

Data  To  Be  Gathered 

"Grades  for  Mr.  Smith's  class  and  the  grades  for  Mrs. 
Jones  class  before  they  switched  the  tests. 

The  grades  for  Mr.  Smith's  and  Mrs.  Jones'  class  on 
the  tests  they  switched. 

The  grades  on  the  other  test  they  took." 

(Unlike  most  of  the  responses,  this  student  was  able 
to  answer  with  a  quantifiable  hypothesis.) 

Hypothesis 

"Mr.  Smith  is  not  as  good  a  teacher  as  Mrs.  Jones. 


Design  of  Experiment 

"Give  both  teachers  a  certain  amount  of 
time,  say  two  weeks,  to  teach  their  class 
about  a  certain  time  period  in  English,  one 
that  neither  class  has  ever  studied.  At  the 
end  of  two  weeks,  give  both  classes  the 
same  test  on  the  time  period.  You  might 
also  give  each  teacher  a  questionnaire  to 
see  how  he/she  feels  about  his/her  career." 

Data  To  Be  Gathered 

"A.  How  many  people  are  in  each  class. 

B.  How  long  each  teacher  has  been 
teaching. 

C.  The  test  grades  of  each  class. 

D.  The  averages  of  everyone  in  the  class 
for  the  past  few  years. 

E.  Review  the  results  of  the  question- 
naire, maybe  Mr.  Smith  does  not 
enjoy  being  a  teacher." 

(Although  the  student  is  on  shaky 
ground  with  a  hypothesis  that  is  depend- 
ent upon  a  value  judgement  i.e.,  what 
constitutes  a  good  teacher;  this  student 
controls  for  the  effect  of  prior  knowledge 
and  the  level  ofdiffiadty  of  the  test.  The 
questionnaire  gives  attttudinal  infor- 
mation that  cannot  be  controlled  for  but 
which  might  explain  the  results.) 


**  27%  Hypothesis  is  less  clear/feasible, 
but  experiment  addresses  an 
appropriate  hypothesis;  minor 
weaknesses  in  instrumentation 
or  control  of  variables. 

Hypothesis 

"Mrs.  Jones  is  a  better  teacher." 


Design  of  Experiment 

"About  5  students  are  switched  from  Mr.  Smith's  class 
to  Mrs.  Jones'  class  and  vice  versa.  The  students  spend 
one  quarter  of  the  year  in  their  new  class  and  are  told 
to  try  their  hardest.  Then  they  are  switched  back  for 
another  quarter  of  a  year  and  are  again  told  to  try  their 
hardest." 

Data  To  Be  Gathered 

"Compare  the  marks  of  the  ten  students  in  each  class. 
If  all  ten  have  tried  their  hardest  and  passed  in  the 
same  quality  of  work  and  still  receive  different  grades, 
then  Mr.  Smith  is  a  harder  grader.  If  the  students  try 
their  hardest  and  pass  in  lower  quality  of  work  in  Mr. 
Smith'  class,  then  maybe  Mr.  Smith  is  not  as  good  a 
teacher  or  is  harder." 

(This  response  is  very  similar  to  the  previous 
response  but  the  student  does  not  account  for  the 
abilities  of  the  students.  There  is  a  great  emphasis 
on  the  intangible  of  "try  their  hardest ") 


*  11%  Somewhat  vague  experiment;  seems  to  address 
hypothesis;  does  not  necessarily  account  for 
some  variables. 

Hypotliesis 

"Mr.  Smith's  English  class  isn't  as  smart  as  Mrs.  Jones 
English  class." 

Design  of  the  Experiment 

"I  would  give  both  classes  about  one  week's  worth  of 
the  same  work  and  see  which  class  would  get  the  better 
grades." 

Data  To  Be  Gathered 

"I  would  gather  the  work  that  they  would  do.  Make 
them  do  it  and  get  an  average  of  their  grades  when  is 
was  over  and  see  who  was  smarter." 

(Compare  this  response  with  the  response  in  the 
first  category  where  the  student  collected  the  class 
averages  as  a  basis  for  comparison.  This  student 
does  not  account  for  differences  between  the 


teachers  or  that  grades  are  a  function  of 
the  teacher,  nor  does  the  student  define 
the  word  smart  The  only  item  control- 
led is  the  material  covered  during  the 
fixed  period  of  time.) 


*** 


very  good  answer 
**    good  answer 
*    minimally  correct  response 


Incorrect  Answers 

21%  Hypothesis  weak  or  not  testable; 
experiment  does  not  address 
hypothesis  or  addresses  it  poorly; 
collects  poor  information;  or  no 
real  experiment  or  investigation 
proposed. 

Hypothesis 

"The  students  are  not  interested  in  the 
class." 

Design  of  the  Experiment 

"Ask  questions  to  teachers  and  students 
about  homework  and  classwork,  etc.  and 
ask  if  students  were  participating." 

Data  To  Be  Gathered 

"Answers  to  above  questions  and  more 
hypotheses." 

(This  response  does  not  generate  a  tes- 
table hypothesis  nor  does  it  control  for 
any  variables  that  might  cause  differen- 
ces in  the  grades  of  the  class  nor  does  it 
measure  interest  except  by  question- 
ing.) 


3%  Gives  recommendations  for  improving  test 
scores. 

Hypothesis 

"They  weren't  trying  as  hard." 

Design  of  Experiment 

"Let  them  come  after  school  to  get  extra  help." 

Data  To  Be  Gatliered 

"See  how  much  they  have  increased  their  grades." 

3%  Other  incorrect  response. 
4%  Blank 


Comments 

Sixty-eight  percent  of  the  students  were  able  to  answer  this  ques- 
tion with  some  success  indicating  that,  by  grade  eight,  students 
have  a  working  knowledge  of  how  to  generate  a  hypothesis,  how 
to  devise  an  experiment  to  measure  it,  and  how  to  recognize  and 
prioritize  the  data  collected.  This  was  a  difficult  question  because 
of  the  intangibles  in  dealing  with  human  behavior.  Many  students 
seemed  to  recognize  this  and  dealt  with  it  appropriately  This  may 
be  considered  more  of  a  social  science  type  of  experiment  because 
the  difference  in  perspective  does  not  lend  itself  to  traditional 
scientific  analysis.  Wherever  it  is  categorized,  the  same  require- 
ments for  experimental  design  would  apply  Because  a  social 
scientist  observes  diflferent  phenomena,  the  rules  of  measurement 
don't  change;  only  the  instruments  change. 

One  of  the  most  important  aspects  of  science  that  students  learn 
in  their  early  and  middle  years  is  accurate  measurement.  When 
deciding  what  data  is  important  to  gather,  students  must  also 
decide  how  to  measure  it  accurately,  how  to  check  those  measure- 
ments and  how  to  evaluate  the  results  of  those  measurements. 
This  collection  and  use  of  evidence  is  the  basis  of  all  scientific 
investigation.  At  the  early  grades,  students  can  measure  the 
concrete.  In  the  middle  years,  students  can  begin  to  deal  with 
more  ambiguous  situations  in  preparation  for  the  needs  of  higher 
level  science. 
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Students'  Responses  to  Open-Ended 
Questions  in  Science 


Grades  8  &  1 2 


Correct  Answers 


The  moon  orbits  the  earth.   Explain  why  the  moon 
doesn't  fall  to  earth. 


Major  Concepts/Abilities  Tested 

Physical  science 

IVIotion 

Gravity/inertia 


The  percentage  of  student  responses  and  appropriate 
examples  are  listed  under  each  of  the  categories  below. 


Gr8Grl2 


***    0%     3% 


Mention  of  vectors  and 
resultant  motions  caused  by 
inertia  and  gravity  with 
e]i^lanation  or  drawing. 

"The  moon  has  both  angular  and 
linear  velocities.  While  the  moon 
orbits  the  earth,  the  earth  pulls 
down  on  it  with  the  force  of  gravity. 
The  moon  fights  the  pull  of  gravity 
and  wants  to  fly  away  from  the 
earth.  The  moon  stays  in  orbit  and 
moves  with  a  velocity  tangent  to  the 
elliptical  orbit  (vector  diagram  in- 
cluded)." (grade  12) 


**  6%  12%  Answer  discusses  two  dimen- 
sional motion  resulting  from 
inertia  and  gravity. 

"The  moon  is  kept  in  orbit  by  the 
gravity  of  the  earth  and  the  moon 
wanting  to  go  off  in  a  straight  line. 


Massachusetts 


Educational 
Assessment 
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Massachusetts  Department  of  Education 


April  1991 


In  the  spring  of  1990  over  9,000  fourth,  eighth,  and  twelfth  grade  students  were 
assessed  using  open-ended  mathematical,  scientific,  social  studies,  and  reading 
concepts.  This  series  of  reports  describes  and  discusses  the  results  of  these 
assessments.  Prepared  by  Brenda  Thomas,  Elizabeth  Badger,  and  Patrick  Markham. 


*  10%  11% 


*** 


The  two  opposing  forces  keep  it  in  a  circle 
around  tlie  earth."  (grade  8) 

(The  student  mentions  two  separate  motions 
but  does  not  indicate  vectors.  This  response 
is  weaker  than  one  predicated  on  vector 
analysis  but  it  does  indicate  a  high  degree  of 
comprehension.) 

Correctly  describes  either  the  affect 
of  gravitational  force  or  the  inertial 
motion  of  the  moon. 

"The  moon  is  extremely  distant  from  the  earth 
thus  the  force  of  gravity  t)etween  them  is  weal< 
in  comparison  to  the  force  between  the  earth 
and  ourselves.  The  force  of  the  gravitational 
pull  is  strong  enough  to  keep  it  in  orbit  but  not 
strong  enough  to  pull  it  down."  (grade  12) 

"The  moon  maintains  its  orbit  because  its  mass 
is  so  great  it  isn't  pulled  in.  Plus  the  circular 
orbit  keeps  it  moving  in  space."  (grade  8) 

(Although  this  example  contains  some  flaws, 
e.g.,  a  large  mass  would  increase  gravita- 
tional attraction  and  orbit  does  not  cause 
the  movement,  the  student  indicates  an  un- 
derstanding that  the  moon 's  orbit  is  depend- 
ent on  one  of  the  two  factors — the  moon's 
inertia. 


very  good  answer 
**    good  answer 
*    minimally  correct  answer 


Incorrect  Answers 

13%  12%    Identifies  TWO  OPPOSING  FORCES  but 
not  correctly 

"The  moon  remains  away  from  the  earth  be- 
cause the  outer  force  of  the  universe  acts  upon 
it.  The  earth's  gravity  balances  this  force  so 
that  the  moon  remains  in  orbit."  (grade  12) 


36%  26%    GRAVITY  with  incorrect 

explanation  of  this  one  force. 

"The  moon  does  not  fall  to  earth 
because  of  gravity,  an  electromag- 
netic field  which  balances  the 
universe."  (grade  12) 

5%     9%    The  moon  is  in  its  orbit/place/ 
position. 

12%  12%    Response  suggests  there  is  no 
gravity  in  space. 

12%  11%    Other  incorrect  response. 

5%     7%    Blank 


Comments 

The  lack  of  growth  between  grades  eight  and  twelve  is 
particularly  noticeable  in  the  responses  to  this  ques- 
tion. Those  students  who  answered  gravity  were  on  the 
right  track;  they  seem  to  have  some  inkling  of  the 
relationship  between  gravity  and  motion,  but  could  not 
reason  the  relationship  between  the  two  motions 
producing  a  resultant  motion;  instead  they  arrived  at 
a  convenient  explanation  as  to  why  the  moon  didn't 
fall  to  earth  due  to  gravitational  attraction.  This  is  in 
line  with  research  indicating  that  students  often  lack 
a  real  understanding  of  scientific  concepts.  The  mean- 
ing of  a  term  becomes  attached  to  the  particular 
context  in  which  it  was  taught  and  does  not  transfer  to 
other  relevant  situations.  The  contradictions  that 
occur  are  often  unrecognized  by  the  student  or 
teacher. 

The  principle  entailed  in  the  interaction  of  gravity  and 
inertia  is  universal.  It  can  be  emphasized  in  many 
forms  across  the  teaching  of  science  so  that  suidents 
who  understand  the  principle  of  falling  bodies  on  earth 
can  apply  that  principle  to  the  earth  and  the  moon. 
Responses  to  this  question  imply  that  students  are  not 
looking  at  a  principle,  but  rather  at  a  discrete  piece  of 
content  that  has  no  application  other  than  in  answer- 
ing this  test  question. 
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